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Abstract _In this research, novel algorithm based on the optimal 

control theory is proposed for Samarai micro aerial vehicle 
optimal guidance policy. Considering wind effect in the system 
dynamic equation increase the robustness of this optimal 
guidance. Open-loop optimal control is obtained regarding to 
novel and new proposed method. In this way, intelligent 
methods such as GA-PSO optimizer and neural-fuzzy are 
utilized in the proposed new algorithm. Results of new 
algorithm are compared with pseudo-spectral optimal control 
solver and show high accuracy. Closed-loop guidance not only 
damped noises but also simplify controller performance for this 
unstable vehicle. Next, closed-loop optimal guidance base on 
neural-fuzzy method is proposed to achieve autonomous 
guidance to increase the stability of this unstable micro vehicle 
versus wind effects. 

Keywords _ Optimal Control; Wind Effect; Autonomous 
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L INTRODUCTION 

Rescue vehicles for earthquake to get information from 
damages or generally civil applications of unmanned air 
vehicles have increased in recent years. Small unmanned air 
vehicles (SUAV) are used in civil projects where the focus is 
on autonomous vehicles; however, MAVs like Samarai make 
use of radio controls [1]. Lockheed Martin's Intelligent 
Robotics Laboratories has spent the last five years to develop 
an unmanned micro aerial vehicle to replicate the motion. The 
idea was based on maple seed; the seeds that drop from maple 
trees, whirling softly to the ground like silent one-winged 
helicopters, Therefor, these air vehicles are the inspiration for 
a new kind of flying machine that could be useful for military 
and civil information-gathering missions. Lockheed Martin 
Advanced Technology Laboratories (ATL) developed the 
Samarai MAV, a 30- centimeter-radius maple seed like 
aircraft that can take-off/land vertically and fly laterally (like 
a helicopter) to the intrinsic stability of nature’s maple seeds 
[2]. Dynamic systems solutions in the optimal control 
framework can be classified as the two main categories of 
open-loop and closed-loop. Open-loop solution is proper, if 
there are no un-modeled disturbances and/or process noises. 
Unlike open-loop optimal controls, closed-loop optimal 
controls are considered as functions of states. 
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Hence, closed-loop optimal controls increase robustness of 
the vehicle against noises and/or undesirable disturbances 
exerted on the system. Of course, it is usually difficult to 
achieve closed-loop optimal controls. Therefore, finding a 
method with robust characteristics to overcome this difficulty 
is highly desirable. Neural-fuzzy combined with the highly 
transparent and compact form of fuzzy rules makes dynamic 
systems candidate against disturbances and/or noises as 
closed-loop method for autonomous vehicles [3, 4]. 

Miliary and Zachari applied the theory of partially Markov 
decision processes to design guidance algorithms for the 
motion of unmanned aerial vehicles. They used on-board 
sensors for tracking ground targets [5]. Han and Bang 
investigated proportional navigation guidance to avoid 
collision based on the optimal method [6]. In 2006, 
autonomous operation was demonstrated for unmanned air 
vehicle by Ma and Stepangan [7]. Also, autonomous guidance 
system based on receding horizon optimization was described 
by Mettler and Dadkhan [8]. In 2010, Paw and Balas 
presented an integrated framework for small unmanned aerial 
vehicles’ flight control development. Moreover in reference 
[9], software-in-the-loop and flight testing are conducted with 
a synthesized controller [9]. There has not been complete 
investigation into MAVs like Samarai with one wing and 
radio controller that made by Lockheed Martin's Intelligent 
Robotics Laboratories [2]. Therefore, Kellas investigated the 
design and development of controllable single-blade 
autorotation vehicles in 2007. Simulation results were 
examined to provide insight into selecting the best control 
concept and hardware for the final guidance of Samarai 
design. The free-flight simulation results predicted 
approximately 10% of experimentally observed performance 
while the coning angle was predicted to be 25% of the 
observed angle [1]. 

An efficient strategy was proposed by Babaei and Mortazavi 
to design the autopilot for a UAV which was non-minimum 
phase, and its model included both parametric uncertainties 
and unmodeled nonlinear dynamics. Babaei’s work had been 
motivated by the challenge of developing and implementing 
an autopilot that was robust with respect to these 
uncertainties. By combination of classic controller as the 
principal section of the autopilot and the fuzzy logic 
controller to increase the robustness a new methodology was 
developed [10]. 

Challenges uniquely associated with developing this type of 
vehicle are identified and a dynamic modeling and control 
synthesis procedure [11]. Therefore, this paper focuses on the 
optimal control and designing closed-loop trajectories. 
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This paper describes a principled framework to design a novel 
algorithm to guide a MAV for civil applications. Also, this 
research suggests a neural-fuzzy guidance based on the 
trained optimal trajectories from open-loop solutions. Thus 
optimal trajectories are achieved through optimal control 
theory with a novel algorithm and GA-PSO optimizer for 
many scenarios, to train neural-fuzzy method. In this way, 
new system dynamic is utilized to consider wind effects for 
increasing the robustness of the mentioned unstable vehicle. 



II. BRIEF REVIEW TO THE OPTIMAL CONTROL 
THEORY 

Dynamic optimization of dynmic system may be considered 
as optimal control theory. Calculus of variation is the most 
important method in the optimal control theory. Optimal path 
or value of the control variable is the key point in the optimal 
control solution. In other hand, if the control variable is found, 
the optimal solution for the state variables are derived. 
Optimal control problem can be stated in the complex vector 

form X —f (x,U,t) to minimize or maximize cost 

function like J=^L(t,x)dt where optimal controller 

o 

belongs to time domain [0, T]. T is the final time of optimal 
process and it may be free or fixed based on the problem. 
Also, for optimal control problems boundary conditions such 
as initial conditions or final conditions are emphasized as 

x(t 0 )=x 0 and x(t f )=x f or X f =free . Above 

discussion yields to the two point boundary value problem 
(TPBVP). Many methods are introduced to solve TPBVP [3]. 
However, in this work based on the basic phenomena of the 
optimal control theory, new method is introduced to solve the 
problems of these field and overcome difficulties of usual 
method such as variation of external, shooting method, 
pseud-spectral method and etc. 

III. NOVEL METHODTO SOLVE OPTIMAL 
CONTROL PROBLEMS 

In the optimal control theory m equations are derived with 
respect to the dynamic of the main problem to illustrate the 
behavior of the system, however, the optimal dynamic system 
encounters with more than n unknown. These unknown 
parameters are optimal controls. In this novel algorithm, 
system equation is re-derived based on variation of optimal 


control with respect to time. Therefore, number of equations 
and unknown parameters will be equal and one can solve this 
system equation as ordinary system equations with initial 
boundary conditions. In this novel method that proposed, 
optimal control derivatives are obtained by new method to 
construct new system dynamic 


*1 =fi(x,t,u) 

2^.X dNew Method 


x x =f l (x,t,u) 
x 2 =f 2 (x,t,u ) 


X n =f„(x,t,u) 

main system dynamic 


X n =f n (x,t,u) 


du . 

— = g(x,t) 
At 

New System dynamic 


( 1 ) 


du 

An important point is "How - is obtained based on 

dt 

optimal control theory". The second point in this new method 
belongs to achieve suitable conditions for these augmented 
du 

equations for integrating-. When these two mentioned 

dt 


questions are answered, this new method for solving optimal 
control problems is achieved. To answer these two equations 

du - u \ 

about achieving - and U 0 (t 0 ), new mathematical method 

dt 


is proposed an considered the mathematical new phenomena. 
Solution of the first question belongs to mathematical 
methods. Therefor at first, states and co-state relations of the 

^ dH y dH 

system dynamic are achieved x = —— and A — - 

dA dx 


where x is state vector, X is co-state vector and H is the 
Hamiltonian of the system. Next when co-states are achieved 
(without integration) in this method, one can use derivative of 

A(x ,u ) with respect to time. Achieving co-states without 


integration is belonged at the optimality condition like 

dH . dH n 

—— = U . From —— = U one can obtain n co-state of the 

du du 

system analytically. It should be noted that n is the number of 

,, d /L dA. d /L 

control variables. So d A t = —— {dx) -\ -— {du ) -I- 


dx 


du 


dt 


d/ L dA. .dx. d/l .du. d /L 

and - - = —y- (-) -I-(-) -I- - in this way 

dt dx dt du dt dt 

d A. dA. .dx . dA. 

— 1 -—) + —- 

dt dx dt dt 


du du 

- is achieved as - 

dt dt 


dA . 
du 


and new equations for dynamic system in this new method is 

d A. dx 

obtained. It should be noted-, - are achieved form 

dt dt 

the basic phenomena of the optimal control theory. Also when 

dA. 

co-states are achieved analytically in this method —— and 

du 
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dA. 

dx 


are derived simply. Moreover when dynamic system 


dH 


— = V cos(<p)cos(y/) + u 
dt 

dy 


isn't dependent of time explicitly-= 0 . The next question — = V cos ( <p ) sin{y /) + 


dt 

is about initial conditions for this new system equations. 

du 

Hence, these unknown conditions for equations (-) are 

dt 

achieved by intelligent methods. Therefor optimization 
algorithms are used to obtain these unknown conditions with 
respect to minimize or maximize the main cost function based 
on optimal control criterion. Finally, when new augmented 
equations based on mathematically the derivative of the 
optimal controls are obtained and also their unknown initial 
conditions are obtained by optimization algorithm, new 
optimal system dynamic is achieved and it can be solved 
analytically or numerically simply. Bellow chart illustrates 
the main idea of this new method to simplify understanding 
mentioned novel algorithm. 

Dynamic system 


( 2 ) 


dt 

— = V sin((p) + w 
dt V ’ 

Where u, v, w are non-dimensional wind velocity as 

X y Z 

u = —, V = —, w = — and V is relative velocity of 
X V 7 

W -S w 

Samarai. Also X , V and z are non-dimensionlized 

parameter regarding to wind characteristic. To construct 
optimal control solution, Hamiltonian of the system should be 
obtained. 

H = L + A X (V cos((p)cos(y/) + u ) 

+ A y {V cos[(p)sin[y/) + v ) (3) 

+ 2. (V sin(<p) + vu) 

Main cost function is considered as minimum-time so, L = 1. 


Optimal control theory 
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Fig. 1, Main chart of the novel algorithm 

IV. INVESTIGATING OPEN-LOOP SOLUTIONS 

To obtained results and derived equations for co-states, last 
discussions about achieving new system dynamic are 
considered. Hence, the system dynamic for three dimensional 
motion with respect to wind effects are as bellow. 


Also, formulas for obtaining co-states equations regarding to 
optimal control theory are: 

dl dH 2 


dt 

dA- 


dH 

dx 


A 

y w 

a ,, 


(4) 


__dH 
dt dy 

dA^__dH__ __ 

dt dz z w 

Regarding to optimal controls as two angles (£>(t ),y/(t) 
optimality conditions are introduced as follows: 

dH 

dtp 

dH 


= 0 


(5) 


= 0 


dy/ 

Based on the above equations, one can obtain two unknown 
variables such as two co-states analytically. Moreover, for 
min-time cost function that final time is free and explicit terms 
for time are not seen in the system equations the Hamiltonian 
is equal to zero. Hence, three co-states can be determined 
such as: 


A = 


-x 


( 6 ) 


V (ux + v y + wz ) 

a ~y 

y V(ux + vy + wz ) 

2, =—-—-- 

V[ux + vy + wz ) 

Now, proposed algorithm is utilized based on analytical 
solutions for co-states. Therefore, with respect to bellow 
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dcp dif/ , . , 

equation, - and - are obtained to construct new 

dt dt 


optimal system equation regarding this novel algorithm. 

dA t dA t dx 

— _ JL _ Sx dt i = n 

dt dZ X ’ y ’ Z (7) 


du 


du 

The key point of the mentioned method is referred to initial 
and final conditions for new augmented equations (see above 
equation) that tuned by optimizer to maximize or minimize 
the main cost function. In this problem, main cost function is 
considered as min-time criteria. In the next part, simulations 
and results are sketched to show the accuracy of the work. At 
first, comparison between this method and pseudo-spectral 
method is investigated to valid the results of the new 
algorithm. 



t(sec) 



Fig. 5, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle), three dimensional 



Fig. 6, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle) for the first controller (deg) 


Fig. 2, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle) for z direction (meter) 




Fig. 7, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle) for the first controller (deg) 

V. AN INTRODUCTION TO FUZZY SYSTEMS 


Fig. 3, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle) for x direction (meter) 



Fig. 4, Comparison Novel Algorithm (line) and pseudo-spectral 
method (circle) for y direction (meter) 


Fuzzy systems are accurately-defined systems. Although 
fuzzy systems describe uncertain and unspecified phenomena, 
the fuzzy theory itself is an accurate theory. In operational 
systems, important data are resulted from two sources. One 
source is the expert whose knowledge and wisdom on system 
is defined using a natural language. Another source is the 
mathematical model and measurement emanating laws. 
Hence, combining these two types of information is an 
important issue in systems design. A Fuzzy system is 
composed of four sections including recognition database or 
fuzzy rules, fuzzy-maker unit, decision-making or deduction 
unit and fuzzy-remover unit. Input signals are converted into 
fuzzy language variables in the fuzzy-maker unit. Then 
system output is produced in the fuzzy form by 
decision-making unit which uses the existing rules and 
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combines them with recognition database information. 
Finally, the output passes through the fuzzy-remover unit and 
becomes quite non-fuzzy. 

a More advanced method, neural-fuzzy training method was 
used to overcome the weaknesses of the fuzzy method. These 
weaknesses could be enumerated as follow: 

> Limited rule definition 

> Limited membership function over the specified range 

> Non optimal membership functions 

Noteworthy is that, neural-fuzzy method could be used to 
define optimal membership functions. In other words, smart 
neural logic tries to optimize the specified range of the 
membership functions leading to reduced errors in definition 
of the membership functions which itself contributes to 
optimization. Sugeno deduction rule has been used in the 
neural-fuzzy training in this work where the number of inputs 
and outputs are 3 and 1, respectively for states and divided 
d(p 

controls - . As it is clear from these figures, behavior is 

dy 

highly complex and quite nonlinear denoting the fact that it is 
a complicated and difficult task to train a smart system with 
such non-linear behaviors for autonomous optimal guidance. 



Fig. 8, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle) for z direction (meter) 



Fig. 9, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle) for y direction (meter) 



Fig. 10, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle) for x direction (meter) 



Fig. 11, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle), three dimensional 



Fig. 12, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle) for th q first controller (deg) 



Fig. 13, Comparison Novel Algorithm (line) and neural-fuzzy method 
(circle) for the second controller (deg) 

In the next part to examine the robustness of optimal 
neural-fuzzy guidance, noises are exerted on the system. 
Therefore, from figures it is concluded that autonomous 
optimal guidance policy can direct samarai with respect to 
noises such as navigation system. 



Fig. 14, Comparison open-loop solution and closed-loop solution 
with noise, z direction (meter) 
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Fig. 15, Comparison open-loop solution and closed-loop solution 
with noise, y direction (meter) 

Above figure show the higher impact is acted on y direction 
and red line is open-loop solution with noise that 
non-autonomous system can not damp the exerted noises. 



Fig. 16, Comparison open-loop solution and closed-loop solution 
with noise, x direction (meter) 



Fig. 17, Comparison open-loop solution and closed-loop solution 
with noise, three dimensional 



Fig. 18, Comparison open-loop solution and closed-loop solution 
with noise, the first optimal controller (deg) 



Fig. 19, Comparison open-loop solution and closed-loop solution 
with noise, the second optimal controller (deg) 

VI. CONCLUSION 

In this paper, new and novel algorithm based on the optimal 
control theory is achieved for Samarai micro aerial vehicle 
autonomous guidance policy. In this way, wind effects in the 
system dynamic equations are considered to increase the 
robustness of this optimal guidance. In this work not only 
open-loop guidance but also closed-loop guidance is obtained 
by new method based on the intelligent methods. Intelligent 
methods such as GA-PSO optimizer and neural-fuzzy are 
utilized in the proposed new algorithm. Results of the new 
algorithm are compared with pseudo-spectral optimal control 
solver to show high accuracy of results against wind effects 
and noises. 
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